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Abstract — A set of m terminals, observing correlated signals, 
communicate interactively to generate common randomness for 
a given subset of them. Knowing only the communication, how 
many direct queries of the value of the common randomness 
will resolve it? A general upper bound, valid for arbitrary signal 
alphabets, is developed for the number of such queries by using 
a query strategy that applies to all common randomness and 
associated communication. When the underlying signals are in- 
dependent and identically distributed repetitions of m correlated 
random variables, the number of queries can be exponential in 
signal length. For this case, the mentioned upper bound is tight 
and leads to a single-letter formula for the largest query exponent, 
which coincides with the secret key capacity of a corresponding 
multiterminal source model. In fact, the upper bound constitutes 
a strong converse for the optimum query exponent, and implies 
also a new strong converse for secret key capacity. A key tool, 
estimating the size of a large probability set in terms of Renyi 
entropy, is interpreted separately, too, as a lossless block coding 
result for general sources. As a particularization, it yields the 
classic result for a discrete memoryless source. 

Index Terms — Common randomness, Gaussian secret key ca- 
pacity, interactive communication, query, query exponent, secret 
key capacity, strong converse. 



I. Introduction 

A set of terminals observing correlated signals agree on 
common randomness (CR), i.e., shared bits, by communicating 
interactively among themselves. What is the maximum number 
of queries of the form "Is CR = IT' with yes-no answers, that 
an observer of (only) the communication must ask in order to 
resolve the value of the CR? As an illustration, suppose that 
two terminals observe, respectively, n independent and identi- 
cally distributed (i.i.d.) repetitions of the finite-valued random 
variables (rvs) Xi and X2- The terminals agree on CR X'l 
with terminal 1 communicating to terminal 2 a Slepian-Wolf 
codeword of rate H {Xi \ X2) obtained by random binning. 
An observer of the bin index can ascertain the value of CR with 
large probability in approximately cxp [nl {Xi A X2)] queries 
(corresponding to bin size). Our results show that more queries 
cannot be incurred by any other form of CR and associated 
interactive communication. 

In a general setting, terminals 1, m observe, respectively, 
n i.i.d. repetitions of the rvs Xi, Xm, and communicate 
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interactively to create CR, say L, for the terminals in a 
given subset A C {!,..., m}. For appropriate CR L and 
communication F, the number of queries of the form "Is 
L = 17' that an observer of F must ask to resolve L is 
exponential in n. We find a single-letter formula for the largest 
exponent E*. Remarkably, this formula coincides with the 
secret key (SK) capacity for a multitermial source model 
with underlying rvs Xi,...,Xm S, ifTol . The latter is the 
largest rate of nearly uniformly distributed CR for A that 
meets the security requirement of being nearly independent 
of the communication used to generate it. While it is to be 
expected that E* is no smaller than SK capacity, the less- 
restricted E* may seem a priori to be larger But it is not so. 
The coincidence brings out, in effect, an equivalence between 
inflicting a maximum number of queries on an observer of F 
on the one hand, and imposing the explicit secrecy constraint 
above on the other hand. In fact, as in the achievability proof 
of SK capacity in ||9l, the exponent E* is achieved by the 
terminals in A attaining "omniscience," i.e., by generating 
CR L ~ for A, using communication F of 

minimum rate. 

Alternatively, E* can be interpreted as the smallest rate of a 
list of CR values produced by an observer of F which contains 
L with large probability. 

Our main contribution is a new technique for proving 
converse results involving CR with interactive communication. 
It relies on query strategies for L given F that do not depend 
explicitly on the form of L or F, and do not require the 
rvs {Xit, Xrat)^^i to be finite-valued or i.i.d. In fact, our 
converse results hold even when the underlying alphabets 
are arbitrary, but under mild technical assumptions. Jointly 
Gaussian rvs are treated as a special case. Furthermore, our 
converses are strong in that the characterization of E* does not 
depend on the probability of recovery of the CR. This, in turn, 
leads to a new strong converse result for the SK capacity of 
the multiterminal source model ||9], |[T0) . A byproduct of our 
technique is a simple lossless block coding result for general 
finite sources, in terms of Renyi entropies. A particularization 
recovers the classic lossless block coding result for i.i.d. 
sources [l24l without recourse to the asymptotic equipartition 
property (AEP). 

The number of queries above can be interpreted as a 
measure of the correlation among the random signals observed 
by the terminals: A stronger correlation necessitates more 
queries for resolving the CR that can be generated by them. 
Such a measure of correlation is in the spirit of the body of 
work on "guessing" the value of an rv based on a correlated 
observation ||2l], E, Hll. 
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The problem formulation and our main result characterizing 
the optimum query exponent are given in the next section. 
Simple and essential technical tools which also may be of 
independent interest are presented in Section |lll] Achievability 
is proved in Section |IV] The less complex converse proof for 
the case A~ {1, ni} is given in Section [V] However, this 
proof does not extend to an arbitrary A C {1,...,™}, for 
which a different converse is provided in Section |VT] Section 
IVIII contains the strong converse result for SK capacity. A 
converse for the optimum query exponent for rvs with arbitrary 
alphabets is proved in Section IVIIII with jointly Gaussian rvs 
as a special case. The discussion in Section |IX] includes the 
mentioned lossless block coding result for general sources. 

II. Main Result 

Let Xi,...,Xm, m > 2, be rvs with finite alphabets 
Xi, . . . , Xm, respectively, and with a known joint probabil- 
ity mass function (pmf) V Xi,...,x„,- For any nonempty set 
A ^ M = {1,...,to}, we denote Xa = {Xi, i G A). 
We denote n i.i.d. repetitions of Xm = {Xi, . . . ,X,n) with 
values iuXm ^ XiX...xXmhy X%^ = (Xf, . . . , X;;J with 
values in X^ = A"" x . . . x A"^. Given e > 0, for rvs [/, V, 
we say that U is e-recoverable from V if P {U ^ .f{V)) < e 
for some function f{V) of V. The cardinality of the range of 
the rv U is denoted by \\U\\, and the complement of a set A 
by A"^. All logarithms and exponentials are with respect to the 
base 2. 

We consider a multiterminal source model for generating 
CR using interactive communication. Terminals 1, . . . , m ob- 
serve, respectively, the sequences X", . . . ,X^, of length n. 
The terminals in a given set ^ C wish to generate 
CR using communication over a noiseless channel, possibly 
interactively in several rounds. 

Definition 1. Assume without any loss of generality that the 
communication of the terminals in A4 occurs in consecutive 
time slots in r rounds, where r can depend on n but is finite 
for every n. Such communication is described in terms of the 
mappings 

/ll ; • ■ • 1 flni-j /2I : ■ • ■ : /2m; • ■ • ; frl-j ■ • ■ ; /rm; 

with fji corresponding to a message in time slot j by terminal 
i, 1 < j < r, 1 < i < m; in general, fji is allowed to yield 
any function of X" and of previous communication 

(t)ji = {fki'-k <j, I eM oi k= j, / < i} 

The corresponding rvs are termed collectively as interactive 
communication 

F = {Fii, . . . , Fim, ^21, • • • 7 F2m, . . . , Frl, . . . , Frm}, 

where F = F'^"'(X]i^); the rv corresponding to (pji is 
denoted by Local randomization at the terminals is not 
considered here for ease of exposition. In fact, allowing such 
randomization does not improve our result; see Section HXlB. 

Definition 2. Given interactive communication F as above, 
an rv L = i'"' (X^) is e-common randomness (e-CR) for 



A from F if it is e-recoverable from (X", F), i G A, i.e., if 
there exist rvs Li = (X",F), i £ A, satisfying 

P{L, =L, ieA)>l- e. (1) 

The rv Li will be called an estimate of L at terminal i ^ A. 

A querier observing the communication F wants to resolve 
the value of this CR L by asking questions of the form "Is 
L = IT' with yes-no answers. While queries of this form 
have been termed "guessing" II2TI . ||2l, [S), 1141 . we use the 
terminology "query" since our approach covers a broader class 
of query strategies; see Section HXl-B. 

Definition 3. For rvs U, V with values in the sets h{,V, a 
query strategy q for U given V = v is a bijection q{-\v) : 
U {1, where the querier, upon observing V — v, 
asks the question "Is U = u?" in the g(it|w)* query. 

Thus, a query strategy q for resolving a CR L on the basis 
of an observed communication F = i is an ordering of the 
possible values of L. The terminals seek to generate a CR L 
for A using communication F so as to make the task of the 
querier observing F as onerous as possible. For instance, if 
L were to be independent of F, then the querier necessarily 
must search exhaustively over the set of possible values of L, 
which can be exponentially large (in n). 

Definition 4. Given < e < 1, a query exponent E > {) is, 
e-achievable if for every < e' < 1, there exists an e-CR L = 
L(") (XX, ) foi AC M from communication F = F {X'^) 
such that for every query strategy q for L given F, 

F{q{L I F) > cxp(n£;)) > 1 - e', (2) 

for all n > N{e,e'). The e-optimum query exponent, denoted 
E*{e), is the supremum of all e-achievable query exponents; 
E*{e) is nondecreasing in e. The optimum query exponent E* 
is the infimum of E*{e) for < e < 1, i.e., 

E* = lim£;*(e). 
Remark. Clearly, Q < E* <\og\XM\- 

Condition (|2|i forces any query strategy adopted by the querier 
to have an exponential complexity (in n) with large probabil- 
ity; E* is the largest value of the exponent that can be inflicted 
on the querier 

Our main result is a single-letter characterization of the 
optimum query exponent E* . Let 

B = {BCM:B^%,A(^B}. (3) 

Let h.{A) be the set of all coflections A = {As : B e S} of 
weights < Ab < 1, satisfying 

Ab = 1, i(^M. (4) 

BeB:B3i 

Every A G A(.4) is called a fractional partition of M (see 

Eoi, ma, m, hod. 

Tlieorem 1. The optimum query exponent E* equals 
E* =E*{e)=H{XM)~ max Y XbH (Xb \ Xb^) , 

< e < L (5) 
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Remarkably, the value of E* coincides with the secret key 
(SK) capacity of a multiterminal source model ||9l, ifTOl . The 
latter is the largest rate of a CR A' = A'(Xj^) for A from 
communication F, with K satisfying the "secrecy constraint" 
of ID: 



Lemma 3. Given 7 > and < (5 < 1/2, let the rvs U, V, 
satisfy 



{u,v) : Pijiv {u\v) < 



7 



>l-5. 



\ims,n{K;F) = 0, 

71 

where the security index Sin is given by 



(6) 



Then for every query strategy q for U given V, 

Fiq{U\V)>j)>l-e', 



(11) 



(12) 



s,n{K; F) = log II A'll - H{K \F)^D {Pk,f \\ Pu„if x Pp) , 

(7) 

with Punif being the uniform pmf on {1, || A'||}. In fact, 
the achievability proof of Theorem [T] is straightforward and 
employs, in effect, an SK in forming an appropriate CR L. We 
show that for such a CR L, any query strategy is tantamount to 
an exhaustive search over the set of values of the SK, a feature 
that is apparent for a "perfect" SK with I{K A F) = 0. The 
difficult step in the proof of Theorem [T] is the converse part 
which involves an appropriate query strategy, for arbitrary L 
and F, that limits the incurred query exponents. Our strong 
converse yields a uniform upper bound for E*{e), < e < 1. 

We shall see that while the expression for E* in (|5]l lends 
itself to the achievability proof of Theorem [T] in Section |IV] 
alternative forms are suited better for the converse proof. For 
the latter, denoting 



As 



BeB 



(8) 



the expression (|5]l can be written also as 



E* = mill 
AeA(yi) 



.BeB 



(9) 



which is used in the converse proof for an arbitrary A Ai 
in Section [VT] The converse proof for the case ^ = is 
facilitated by the fact that the right-side of (|9|l can be expressed 
equivalently as i?) (see also |[9l Example 4]) 
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i=l 



(10) 



where the minimum is over all (nontrivial) partitions tt = 
(tti, ...jTTfc) of Ai with |7r| = k parts, 2 < k < m. 

III. Technical Tools 

The following simple observation relates the number of 
queries in a query strategy q to the cardinality of an associated 
set. 

Proposition 2. Let q be a query strategy for U given V = v, 
V GV. Then, 

\{u eU : qiu\v) < 7}| < 7. 

Proof. The claim is straightforward since q{-\v) is a bijec- 
tion. □ 
For rvs U,V, finding a lower bound for q{U\V) involves 
finding a suitable upper bound for the conditional probabilities 
Pu\v (■ I ■)■ This idea is formalized by the following lemma. 



for all e' > 26. 

Conversely, if ( 1721 ) holds for every query strategy q for U 
given V, with < e' < {I - VS)^, then 



iu,v):Pu\v{u\v) < >5. 



(13) 



Proof. Suppose ( fTTT ) holds but not ( fT2] i. Then there exists q 
with 



P(<z(f/|V^) <7) > (14) 



From (dl) and (O 



P v) : Pu\v {u\v) < -, q{u\v) < 7 

>l-5 + e' -l = e' -5. (15) 
On the other hand, the left side of ( fTST i equals 

Ep^(^) E Pu\v{u\v) 

^ u:q{u\v)<j, Pulv{n\v)<^ 

< 7. — , by Proposition 12] 

7 

= S, 

which contradicts ( fTSl l since e' > 26. 

For the converse, suppose that (fTST l does not hold; then, we 
show that a query strategy q^ exists which violates ( fTSI ) when 
< e' < (1 - VS)^. The negation of (US) is 



{u,v) : Puiv {u\v) > 



> 1 - <5, 



which, by a reverse Markov inequalitjQ ifTTl p. 157] (see also 
|fT2l p. 153]), gives a set Vo C V with 

Pv (Vo) >l-V6, (16) 

and 

Pc/iv : P[/|v (u|w) > ^1 i!^>l-V5, veVo- 

(17) 

Denoting by the set {•} in ( [TtI i. we have 

l>P[/|y(W„h)>^, 



so that 



|W„|<7, w e Vo. 



(18) 



The reverse Markov inequality states tliat for rvs U,V with 
P {iU,V) e S) > l~e for some S CUxV, there exists Vo C V such that 
P {{U, V) e S \ V = v) > 1 - V eVo, and P {V eVo) > 1 - 
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For each v G Vq, order the elements of U arbitrarily but with 
the first \Uy\ elements being from Uy. This ordering defines 
a query strategy qo{-\v), v e Vo; for v ^ Vq, let qo{-\v) be 
defined arbitrarily. Then for v G Vo, u G Uy, 



qoiu\v) < 7 



by ([T8]l, so that 



P {qoiU\V) < j) > E Pu,vin,v) 
veVo ueUv 



> 



(19) 



by (O and ([TtI). So, q = qo violates ([T2]i when e' < (1 - 

Vsf. □ 

The next result relates the cardinalities of large probability 
sets to Renyi entropy. The first part is used in the converse 
proofs of Theorem [T] The mentioned result is of independent 
interest. For instance, in Section |IX] it is shown to yield an 
elementary alternative proof of the source coding theorem for 
an i.i.d. (finite-valued) source. 

Definition 5. 1231 Let /i be a nonnegative measure on U. For 
< a 7^ 1, the Renyi entropy of order a of fi is defined as 

I — a ^ — ' 

ueu 

Lemma 4. (i) For every < S < fJ.{U), there exists a set 
Us QIA such that 



M m > fiiu) - s, 



(20) 



and 



lUsl < ,5-"/(i-") cxp (iJa(/i)) , < a < L (21) 

(ii) Conversely, for S, 5' > 0, S + 6' < fJ.{U), any set Us C U 
with ^ (Us) as in t[20}l must satisfy 

\Us\ > (<5')'^*""'^ -6-5') cxp (H^iii)) , a > 1. 

(22) 

Proof, (i) For < a < 1, defining 

Us = ^ugU : fi{u) > cxp [-Ha{fJ.)]j , 

we get 

= fi{Us) + E Z^^'^)- 

1 

u: /i('u)<(5i-" cxp[—Ha{fi)] 

Writing the summand in the right-side above as /i(u) = 
lj,{u)°' fi{uy^^, we obtain 

l^(U) < iJi{Us) + 5 cxp [-(1 - a)H^{ij)] ^ 



ueu 



=^Pl{Us) + 5, 
which is (|20] |. Furthermore, 

exp [(1 - a)Ha{p)] = E P'i'^y 



ueu 



ueUs 

> \Us\6t^ exp [-aHaifi)] , 



which gives ( 1211 1. 

(ii) By following the steps in the proof of (i), for a > 1, it 
can shown that the set 



Uo = ^^ugU : fi{u) < (<5')^^^^ cxp[-i7a(^)]} (23) 



has 



^i{Uo) > fi(U) - 5', 
which, with ( l20b . gives 

fi(UonUs) > ^i{U)-5-5'. 

Since by ^ 

^i{UonUs) < \UonUs\ (<5')'^''""^ exp[-i/,(/i)], 

daii follows. □ 

Finally, the following simple observation will be useful. 
Proposition 5. For pmfs Qi,Q2> on V, 

Qi {{v : Qi{v) > 5Q2{v))) > 1 - 5, < ^ < 1. 
Proof. The claim follows from 

Qi{v)< E 6Q2{v)<5. 

veV:Qi(v)<SQ2{v) veV:Qi(v)<SQ2{v) 

□ 

IV. ACHIEVABILITY PROOF OF THEOREm[T| 

Denoting the right-side of (|5]l by C, we claim, for < e < 1, 
< (5 < 1/2, /3 > 0, the existence of an e-CR L = X']^ for 
A from F with 

P({(a;Xi,i):Pi|F(xX< |i)<5exp[-n(C-/?)]}) >l-<5, 

(24) 

for all n sufficiently large. Then the assertion of the theorem 
follows by applying the first part of Lemma |3] with U = L, 
y = F, 7 = exp[n(C — 13)], to conclude from (fT2l i that 

E*{e) > C, 

since /3 > was chosen arbitrarily. 

Turning to the mentioned claim, it is shown in l|9] Proposi- 
tion 1], llOl Theorem 3.1] that there exists communication F 



such that L = X'^ is e-CR for A from F with 

ilog||F||< max V A^i/ (Xb | ^50) + ^, (25) 



BeB 



for all n sufficiently large. Using Proposition |5] with Qi = Pf 
and Q2 being the uniform pmf over the range of F, we get 



Pf n i : Pf (i) > 



6 



2IIFII 



>1-1 
2 



(26) 



Also, for in the set Tn of Pj^^^ -typical sequences with 
constant 6 ifTTl Definition 2.8], we have 



Px" (xX,) < exp 



nlHiXM) 



(27) 



and 



Pxjt, {Tn)>l-^, 
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for all n sufficiently large. Denoting by 2o the set on the left- 
side of ( [26] l, it follows that 

P (Xli e Tn,F e lo) > I - S. (28) 

The claim results from (l26i-(l2Fi upon observing that for 



Af{xm I i) 



Px" ((a::i,))l(F(x:t,)=i) 



2exp 



PF(i) 

-n (h{Xm) 



< 

d 

< <5exp[-n(C-/3)], 

for all n large enough, where the last inequality is by (IZST l. □ 

Remark. The achievability proof brings out a connection 
between a large probabiUty uniform upper bound k for P^, the 
size j|F|| of the communication F, and the associated number 
of queries needed. Loosely speaking, the number of queries 
is approximately which reduces to if L is nearly 

uniformly distributed. 



V. Converse proof of Theorem[T]for M 

Recalling the expression for E* in ( fTOl i. given a partition 
TT of with \7t\ ~ k, 2 < k < m, we observe that for a 
consolidated source model with k sources and underlying rvs 
Yi, ...,Yk wher^l Yi = . , the e-optimum query exponent 
E*{e) can be no smaller than E*{e) (since the terminals in 
each TTi coalesce, in effect). 

Theorem 6. For every partition t: of M with \tt\ = k, 
Elie) < -^^D (vy, n II W^yX 0<e< 1, 



and so 



E*{e) < min£'*(e) < min ■ 



TT - 1 



-D P 



Theorem |6] establishes, in view of (fTOl i. the converse part of 
Theorem [1] when A = Ai. 

The proof of Theorem |6] relies on the following general 
result, which holds for queries of CR generated in a multiter- 
minal source model with underlying rvs Yi, Yfc for n = 1. 

Theorem 7. Let L ^ L {Yi, ...,Yk) be e-CRfor{l, k}from 
interactive communication F = F (Yi, Yfc), < e < 1. 
Given 6 > such that 5 + \/S + e < 1, let 9 be such that 

Then, there exists a query strategy qq for L given F such that 




P ( qoiL I F) < ( ^ 



(30) 



Proof of Theorem |6] We apply Theorem Q to n i.i.d. 
repetitions of the rvs Yi,...,Yfc. Denoting by 7^' the set of 
Pyi -typical sequences with constant S, we have 

Pn'N...,y," (T;:) > 1 - <5, 
andfor(y?,...,y«)e7;;, 

Pn^...,y,-(?/^-,^/fc) 
UIiPyAv?) 



< exp 



exp 



Y,H{Yi)^H{Yu...,Yk) 



n(D(pY„...,Yj\l[PY. 
^ ^ 1=1 



for all n large enough. Thus, the hypothesis of Theorem |7] 
holds with 



n(D(pY,,...,YjY[PY, 



= tin = exp 



If E is an e-achievable query exponent (see Definition |4]i, then 
there exists an e-CR L ^ L (Yj", Yj!') from communication 
F = F (Y", Y^") such that (O holds for the query strategy 
qo of Theorem |2] for this choice of L and F. In particular for 
e' < {1-6 - VS + ef, we get from ^ and ^ that 

P^cxp(n£;) < qo{L \ F) < ^-^/(fe-i)^ 

fc 



exp 



1 



k-1 



D P 



Yi,...,Yk 



Rp^ 



k - 1 



> (i-s-Vs+lY > 0, 



(31) 



for all 71 sufficiently large. It follows that 

E<j^DfpY,,...,YjY[PY. 
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Since E was any e-achievable query exponent and S > was 
chosen arbitrarily, the assertion of Theorem is established. 

□ 

Proof of Theorem^ Denote by C the set of values of the CR 
L. Using the hypothesis ( [29] l of the Theorem, we shall show 
below the existence of a set To of values of F and associated 
sets £(i) C £, i £ Iq, such that for every i E Iq 



Pi|F (£(i) I i) > 1 - 5 - 



\m\ < 



and Pf (Io) >l-5- ^/e + 5. 



(32) 

(33) 
(34) 



Then, we consider a query strategy (70 for L given F as in 
the proof of converse part of Lemma |3] with L, F, lo, 'C(i) 
in the roles of C/, V, Vo, U^, respectively. Thus, for all i e Iq, 



For specificity, the elements in eacli tt; ai'e an'anged in increasing order. 



go(/ I i) < < 



<52 



6 



and so, as in ( fT9T l, we get by ([32l)-([34li. 



P ( q,{L I F) < ( " ' 



thereby establishing the assertion (l30t . 

The existence of the sets Iq and {£(i),i G Iq} satisfying 
(l32l i-(l34li is argued in three steps below. 
Step 1. First, we note the following simple property of interac- 
tive communication: if rvs Fi, Yfe are mutually independent, 
they remain mutually independent when conditioned on an 
interactive communication F. 



Lemma 8. Let the pmfY'Yi,...,Yk be such that 

k 

Then, for i = F (j/i, j/j.), we have 



(35) 



Pn,...Y.|F (yi, Vk I i) = n Py.lF I i) • (36) 
Proof. The proof follows upon observing that 

/p(r,Ari,...,r,_i,y,+i,...,yfe |F) 

= 0, j = (37) 

where the first inequality is by [1 , Lemma 2.2] upon choosing 
U = Yj, V ^ (Yi,...,Y,_i,rj+i,...,Yfc), $ to be the com- 
munication from terminal j, and ^I* to be the communication 
from the remaining terminals. 

Hereafter in this proof, we shall select 



(38) 



Step 2. In this step, we select the aforementioned set of 
communication values Iq. Let Lj = Lj{Yj,F) denote an 
estimate of CR L at terminal j, j = 1, fc (see Definition 
|2]l. Denote by To the set {•} on the left side of ( |29] l. For each 
realization (/, i) of {L, F), denote by Ai_i C J^i x ... x the 
set 

Ai.i =To n {(2/1, yk) : F (yi, y^) = i, 

(y, , i) = L (2/1, yk) = /, J = 1, fc} . (39) 

Since L is e-CR from F, we have from ^ and ( |29] l that 

p((ri,...,rfe)eAL,F)>i-e-<5. 

By a reverse Markov inequality, there exists a set Ii of values 
of F with 



Pf (ii) > 1 - VTTs, 



(40) 



and 



P{{Yu...,Yk)eAL,F\F^i)>l-V7+5, ieli. (41) 
Next, denote by I2 the set of values of F such that 

<5Pf (i) < Pp (i) , iel2, (42) 



where Pf is, as usual, the distribution of F under P. From 
Proposition |5] with Qi = Pp, Q2 = Pf, we have 



Pf (12) >l-5. 



(43) 



Thus, by gO]) and gSj, Iq = n I2 satisfies ( |34] |. 
S'fe/? 5. In this step, we identify sets £(i) that satisfy ( [32] i and 
(|33] l. For each i G Iq, the sets A; i corresponding to different 
values / are disjoint. Upon defining the nonnegative measure]^ 
fi on C for each i e Xq by 



A^G) =Pn,...,n|F(A,,i I i), leC, (44) 



we get 



A.(£)=^Py,,...,y,|F(Au|i) 

lec 

= p((ri,...,yfc)eyii,i|F = i) 

> 1 - Ve + S, 

by ( I4TI 1. Applying Lemma |4](i) with C in the role of U, we 
set £(i) = Us, and so 



> 1 - (5 - V7+7 



and 



|/:(i)| < exp (i/a(/i)) , < a < L 

It follows from BSi that 



(45) 



(46) 



Pi|F (/:(i) I i) > ^ Py,,...,y,lF(Ai |i) 

-M^(i)) 



> 1 - (5 - 



(47) 



which establishes ( |32] |. 

Finally, we obtain an upper bound on cxp {Ha{iJ,)) for a = 
i, which will lead to ( |33] ). Denote by j C J^j the projection 
of the set Ai i C 3^1 x ... x along the jth coordinate, 
j = 1, fc. The sets Aj ^ are disjoint for different values of 
I, by definition (see (l39ll). Thus, for the pmf PYi....Yk ( l35l l, 
we have 



.tec 



> 



(48) 



E np-.iF(4iiO' 

where the last step follows from Holder's inequality0 lfT6l 
Section 2.7]. Using ( |36] |, the right-side of (|48] | is the same 
as 

k 

EPn...,n|F (4iX...x4i|i)' 

liec 

'Although fj, depends on i, our notation will suppress this dependence. 
"^See 1251 equation (33)] for an early use of Holder's inequality in a CR 
converse proof. 
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which is bounded below by 



Al^ c X ... X Ali. 



Upon noting that Ai^i C To, for all {yi,. 
follows that 



(49) 
(50) 

,yk) e Ai^i, it 



Yi,...,Yk\ 



F (yi 



,yk 



Y^^....Y, 



iVl, ■■■.Vk) 



Pf (i) 



> 



> 



PF(i) 

atiP>-.(2/.) 

Pf (i) 

e Pf (i) 
Pyi,...,Yfc|F(yi,---,2/fc 



where the third equality and the subsequent inequalities are by 
(l38T l. ( |29l ) and ( |42] |. respectively. Combining the observations 
above with ( |48] | and (|49] |, we get 



1 > 



E 



lec 



Pn,...,y.|F(^i.i I i) 

(5-1 e 

k 



Lie£ 



which, recalling Definition |5] further yields 



exp 



L/e£ 



< 



The previous bound, along with ( |46] l. gives (|33T l. 



□ 



VI. Converse Proof of Theorem[T]for arbitrary 
AQM 

The converse technique of the previous section for A = M 
can be extended to an arbitrary C Al, yielding an analogous 
upper bound for E*{e) in terms of divergences. However, the 
resulting upper bound is inadequate as it is known to exceed 
the expression in the right-side of (|9) (see 16]). In this section, 
we develop a new converse technique that targets directly the 
latter 

The main steps of the general converse proof for the case 
^ C are analogous to those in the previous section. The 
central step is the counterpart of Theorem |7] which is given 
next. Given a fractional partition A as in (|4|, its dual partition 

is A = A(A) = {Xb-,B e B} with 

Asc = v^^, BeB, (51) 



A. 



r 



where B is defined in (O and X^^m is given by (|8]l. It is known 
from II20I . and can be seen also from (|4]i and (|8), that 



E 



Arc — 



B£B:B'=3i 



1 




1 


1 






1 


1 





E 

Ea. 



A, 



B£B:B'=3i 



E 



BeB BeB:B3i 

[As™ - 1] = 1, i e M 



(52) 



so that A, too, is a fractional partition of M. 



Theorem 9. Let L = L{Yi,...,Ym) be e-CR for A from 
interactive communication F = F (i^i, y„i), < e < 1. 
Given (5 > such that 5 + \M + e < 1 and a fractional 
partition A G A(^), let 9b^, B Cz B, and 9q be such that 

1 „ , . 1 



P ^^VM ■■ Py^ {vm) < ^, Py« 
>1-S. 



> —,BeB 

'B" 



(53) 



Then, with 



n 

BeB 



'B" 



(54) 



Co 

there exists a query strategy for L given F such that 



P qo{L I F) < 



>(1 



(55) 



where k(6) = {m2"'y"" 6"'+\ 

Proof. As in the proof of Theorem |7] the assertion (l55t will 
follow upon showing the existence of sets Iq and £(i) C £, 
i G Iq, such that ( |32] | and ( l34l l are satisfied, along with the 
following replacement for (l3Jt : 

As™-1 



l^(i)l < 



ie Jo- 



(56) 



To this end, we provide here appropriate replacements for the 
three steps in the proof of Theorem Q 

Step 1. For each B C Ai, consider the pmf Py^ defined by 



P?^(yA4)=Py«(2/i3)Py,. iVB^ 



Note that P^ = P^". The collection of 



pmfs I 



(57) 

P^^ BeB] 



serve as a replacement for the pmf P in 
For the pmf P^ in (|57] |. we note that 



(58) 



since Fkj = fkj {Yj,^kj) and Ygc is independent of Yb 
conditioned on ^^j- The following Lemma serves the role 
of Lemma [8] 



Lemma 10. For B C M and i = F (yjn), we have 

PYs iVB) 



pB 
^Yb 



IF iVB I i) 



11/0=1 rijesPffc.i^fc, I 



(59) 
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where ij^j denotes the past values of communication in i for The following simple extension of Proposition |5] holds: 
round k and terminal j. 



Proof Note that 



^ F 

Pf,v._ (i I yB)PY„ (vr) 

(60) 



;i) 

m r 



Pf|y^(i|2/B)Py,(yB) 



igl-r „ 1=1 k=l 



P§ (i) 



= E I nilP^'-l*- i'ki I I X 
where the previous step is by iSl) . Furthermore, ^^-^Ib 

r 



r m 



Pf|y. (i I Vb) = n HPil^-^,*.. I y^'*^.) 

IT n (^ u ^-) ^ E (n n P-.H*.. I ] X 



fc=i jeS"^ 



n npti*..(*^-.i%)' (61) 



k^ijeB- I 

where the last step uses dH. Next, - ^ '^E 11 11 ^F^A'^^i V^i hfcij | x 

i fc=i 

Pf (i) TT pB 



llPii*.. V^^h., 



{mT^Y^ 5, (65) 



fc=li=l 

/ , . / \ \ where the first inequality is by (|64] |. and (l65t holds since the 

=n n ^Fkj\<i>kj ykj I 11 PFkj|<i>fcj I *fcj) I ■ summand is a pmf for F, as can be seen by directly computing 
k=i \3eB jeB- J A r^ 

(52) the sum. Defining I2 = | ) | ) ^j.B, we get 

j=l BCM 

Then (I6OI 1. along with (IHTT i and (|62T i. gives ( |59] l. Pp (I2) > 1 — (5 

^fep 2. Denoting by To the set {•} on the left-side of i53[ . for 

each L = l,F — i, define The set Iq is defined as Ii D I2, and satisfies ( l34l l. 

S'fe/:' 5. Finally, we define sets £(i) C £, i g Iq that satisfy 
=7^) n {yM ■ F = i, (Ell and (ESJ. For each i e Xq, let 

L,{y,,i)^L{yM) = l,J^A}. (63) /.(0=Py^|F(Ai|i), ^ e (66) 

Analogous to the proof of Theorem |7] the set Ii of values of Then, the sets £(i) satisfying (|32] | are obtained by an appli- 
F with cation of Lemma 2] (i) as in ( |45] | and ( |46] | above. 

The condition (|56] | will be obtained upon showing that for 



P e Ai,F I F = i) > 1 - VTT^, ieXi, x „i 

^sum /i:n\ 

a = , (67) 

satisfies 

it holds that 



r"/(i-")exp(i?„(/.)) < (^— j . (68) 

For j G and B C denote by Ij B the set of i such that 

To do so, first note that for each B E B, the set S'^ n ^ is 
r nonempty. Thus, by ( |63] ). the projections Af^ of Ai i along 

I'^kj i^kj I ) the coordinates in S'^ C are disjoint across I G £. Thus, 

fe=i 

< n Pi=^..i*.. {^k, I *^,) . (64) 1 > n ( E p?i.iF (^M I i 

fc=i seis \/e£ 
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Using Holder's inequality |fT6l Section 2.7], and recalling (fSTT i 
and ([8]) we get 



1 > 



E n p?:ip Ki I i 



(69) 



Next, note from Lemma \T0\ that 



(^ff I i) 



fc=i jeB': 



which, since the order of products can be interchanged, and 
upon using (|64] |. is bounded below by 

r 



It follows that 



BeB 



> 













n 

BeB 


E p^B^(ysO 






n n 

BeB jeB" 


(m2 




ikj 1 ifej) 





(70) 



The right-side of ( fTOl l can be simplified by noting that 



n n 

BeB jeB" 



n 

m2 



A:=l 



r 



Pf (i) , 



(71) 



where the previous step uses (|52t . The definition of To, along 
with dTOl i and (ItTT i. gives 



BeB 



> 



(7712™)" Pf (i) 



I 4^" 



(72) 



Bei3 



Also, since Ai i C To, we have 



which, with ( |54] i and jT^l i. gives 



np?:iF(<i ii 

Bee 
> 











(m2™)"0 









Py^|F(A,,i |i). (73) 



Since A is a fractional partition, lfT9l Corollary 3.4] implies 



> 1, 



(74) 



which combined with (l69l)-(l74li yields 



1 > 



(m2™) 



.lec 



The previous inequality implies (|68l l since 



1 - a 



□ 



VII. Strong converse for secret key capacity 

A byproduct of Theorem [T] is a new result that establishes a 
strong converse for the SK capacity of a multiterminal source 
model, for the terminals in A Ai. In this context, we shall 
consider - without loss of effect - a weaker notion of security 
index than in (|7]i, defined in terms of variational distance: 



w(i^;F) =^Pf (i) 



\\K\\ 

E 

k=l 



K\F 



ik\i) 



1 



lA'll 



(75) 



However, the requirement (|6) on Si„ will be replaced now by 
limnw(i^;F) = 0. (76) 

n 

Definition 6. Given < e < 1, i? > is an e-achievable SK 
rate for ^ C if for every p > 0, there is an = N{e, p) 
such that for every n > N, there exists an e-CR K = K [Xj;^) 
for A from F satisfying 



and 



-\og\\K\\>R-p, 
n 



SvaAK;F) < ^. 

n 



(77) 



(78) 



The supremum of e-achievable SK rates is the e-SK capac- 
ity, denoted C(e). The SK capacity is the infimum of C(e) for 
< e < 1. We recall the following. 

Tlieorem 11. ^ The secret key capacity for A'^ Ai is 

C = E* =H{Xm)- max Y \bH {Xb \ Xg^) . 

< e < 1. 

Remark. The (new) secrecy requirement (l76t is not unduly 
restrictive. Indeed, the achievability proof of Theorem [TT] jO) 
holds with Sin{K]Y) vanishing to zero exponentially rapidly 
in n, which, by Pinsker's inequality (cf. [fTll ). implies ( f76] l. 
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The converse proof in Q was shown under the "weak secrecy" 
condition 



Then, for F i. 



lim -/(X A F) = 0, 



(79) 



which, in turn, is impHed by (l76T l by a simple application of 
Il9l Lemma 1]. 

The strong converse for SK capacity, valid under ( |76t . is 
given next. 



Theorem 12. For every Q < e < 1, it holds that 
C(e) = C. 



(80) 



Remark. It is not known if the strong converse in Theorem [T2] 
holds under ( |79] l. 

Proof. Theorem [TT||]9] already provides the proof of achiev- 
ability, i.e., C(e) > C. The converse proof below shows that if 
R is an e-achievable SK rate, then R is an e-achievable query 
exponent. Therefore, 



R<E*{e) = C, 0<e<l, 



(81) 



where the equality is by (|5]l. Specifically, for every p > 0, 
suppose that there exists K = K {X^ ) and communication F 
satisfying (fTTI i and dTSl l for all n sufficiently large. We claim 
that the hypothesis (fTTT i of Lemma [3] holds with U = K, 
V = ¥ and 7 = exp[7i(i? - 2p)] for every < (5 < 1/2, 
when p is sufficiently small. Therefore, by (fT2] |. i? — 2/3 is an 
e-achievable query exponent which leads to dSTT i since p can 
be chosen arbitrarily small. 

Turning to the claim, observe that 



pQ(fc,i):PK|F(fc|i)> 



exp[7i(_R — p)] 

<pQ(fc,i):Px|F(fc|i)>^ 

<P({(fc,i):|log||i^||PK|F(fc|i)|>l}) 
<E[|log||i^||P;,|F(i^ |F)|], 



where the first and the last inequality above follow from (iTTt 
and the Markov inequality, respectively. 
Next, we show that 



E [| log Ili^llPxiF {K I F) |] < s,ar{K- F) log 



\K\ 



(82) 



Then, the right-side can be bounded above by 

2p\og\XM\, 



p n 
- log - 



(83) 



for all n sufficiently large; the claim follows upon taking n ^• 
00 and p ^ 0. To see (|82] i, note that for ti,t2, \ti — ^2! < 1, 
f{t) = — tlogt satisfies (cf. ifTTl Lemma 2.7]) 



|/(^i)-/(i2)| < ki-t2|log' 



h -U 



(84) 



^PK|F(fc|i)|log||i^||PK|F(fc|i)| 

fc 

= |Paif (fc I i) logP^iF (fc I i) + Pk\f (fc I i) log ||i^|| 



^ ■log II All - ^^IokIIA'I 



k 



k 



\\K\\ " lli^l, 

Fk\f (fc I i) logPxiF (fc I i) - log 

1 



+ 



Pk\f (fc I i) 



K: 



logllA'l 



Pa'If (fc I i) - 



los 



lAl 



Pa'If (fc I i) - 



(85) 



where the previous inequality uses (|84l i with ti = Pk\f (fc I i) 
and t2 = ||Ar|j~^ for every value k of K. Finally, (|82] | follows 
upon multiplying both sides by Pf (i), summing over i and 
using the log-sum inequality ifTTll . □ 
Observe that the proof of Theorem [12] does not rely on the 
form of the rvs K, F, and is, in effect, a statement relating 
the size of any achievable SK rate under the s„cir-secrecy 
requirement (l7&t to the query exponent. As a consequence, 
also the SK capacity for more complex models in which the 
eavesdropper has additional access to side information can 
be bounded above by the optimum query exponent when the 
querier, too, is given access to the same side information. 

VIII. General Alphabet Converse for A = M 

In this section, we present a converse technique for the 
optimum query exponent for rvs with general alphabets, with 
jointly Gaussian rvs as a special case. No corresponding 
general claim is made regarding achievability of the exponent. 
Our technique also leads to a new strong converse for Gaussian 
SK capacity l2l . 

Let yi be a complete separable metric space, with associated 
Borel cr-field <Ti, 1 < i < k; a special case of interest is 
yi = R"'. Denote by y'' the set x ... x J^j. and by a'' the 
product a- fielcfl (Ti X ... X (Tfe on y'^. Let P — PYi,....Yk ^ 
probability measure on (j^''', cr*^). The interactive communica- 
tion {Fji : I < j < r, 1 < i < k} is specified as in Definition 
[T] with the rv Fji taking values in, say, {Zji,^ji), and 
being ct^ -measurable for each fixed value of the preceding 
communication 

<I>j, = {Fst ■■ I < s < j,l < t < k or s ^ j,l < t < i) . 

Then, there exists a unique regular conditional probabil- 
ity measure on {y'^^cr'') conditioned on cr(F), denoted 
PYi,...,Yfc|F (cf- HI Chapter 6]). The notation Qi will be used 
interchangeably for the probability measure PYi,...,Yk\F (' I i)- 

^Hereafter, the terni "product cr-field" of cr-fields ai, ...,o-j., will mean the 
smallest cr-field containing sets from cri x . . . X crj. , and will be denoted, with 
an abuse of notation, simply as cr'^ = cri x ... X crj. . 
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We make the following basic assumption of absolute continu- 
ity: 

Qi « PYi,...,Yk, Pf a.s. in i, (86) 

i.e., ( [86] l holds over a set of i with PF-probability 1. As- 
sumption (|86] ) is satisfied by a large class of interactive 
communication protocols including F taking countably many 
values. Moreover, we can assume the following without loss 
of generality: 



Q, (F-i(i)^) =0, 



dQi 
dP 



Pf a.s. in i, 



(87) 



{y"") = 0, for y"" e F-^(i)^ Pp a.s. in i. (88) 



Next, we define e-CR L from F and its local estimates L^, 
respectively, as rvs taking countably many values, measurable 
with respect to a'' and ai x cr(F), 1 < i < fc, and satisfying 

P {L = L,, 1 < i < k) > I - e. 

The main result of this section, given below, extends Theorem 
|2]to general measures as above. 

Theorem 13. For < e < 1, let L be e-CR from interactive 

communication F. Let P ~ Py^ be a probability measure 

on {y^,a^) with 



P (Ai X ... X Afc) = Jl Py, [A,) A, e cr,, 1 < i < k. 

(89) 

Assuming that P << P, and given (5 > such that 5 + 
Vo + e < 1, let 9 be such that 

p({/:^(/)<^}) >l-'5. (90) 

Then, there exists a query strategy qo for L given F such that 



P I qoiL I F) < ( ^ 



>(1 



(91) 



The proof of Theorem [13] is deferred to the end of this 
section. At this point, we present its implications for a Gaus- 
sian setup. Let X^^"'* be an M" -valued rv, i = l,...,m, and 
MX'j^^ = be jointly Gaussian A/'(0,S(")), 

where S'"^ is a positive definite matrix. We remark that x')^^ 
need not be independent or identically distributed across n. 
The notion of an e-optimum query exponent E*{e), < e < 1, 
is exactly as in Definition H] even though the underlying CR 
now can take countably many values. Also, given a partition 
TT of with |7r| = fc, 2 < fc < m, the quantity -E*(e) is 
defined as in Section W\ 

Proposition 14. For X^j^ - 7V(0, with being 
positive definite, it holds that 



E*{e) < mmE;{e) 

7T 



< e < 1, 



where E^"-* is the covariance matrix of xi^\ i 



and I • I denotes determinant. 

Corollary. When X^ is Lid. in n with Xm ^ A/'(0, E), 

1 . nl:'i|s.j 



E*{e) < min 



2(M-1) 



Ids 



< e < 1. 



Proof. Proceeding as in the proof of Theorem |6] we apply 
Theorem [T3] to the rvs Y, = l<i< |7r|. Specifically, 

we show that the hypothesis (|90] l is satisfied with 




cxp(n(5), 



(92) 



where < S < 1/2 is arbitrary. Then, the Proposition 
follows from the definition of E*{e) and ( |92] i as in the proof 
of Theorem |6] The Corollary results by a straightforward 
calculation. It remains to verify that ( |90] l holds for 9 in ( |92] i. 
For i? C tM, _B ^ 0, let denote the density of the Gaussian 
rv . From the AEP for Gaussian rvs |l8j equation (47)] 
(see also ID), 



-- log gB 
n V 



> T, 



for some 9 ^ B CM 
< 2™exp(-c(r)ri), t > 0, (93) 

where h denotes differential entropy and c(t) > is a positive 
constant that does not depend on n. Since 

dP 



P a.s. 



and 



hiX 



M 



-log(2^e)™|S(")|, 
ilog(2^e)l--l"|l]i'j)| 



1 < i < kl 



using the upper and lower bounds from (193) that hold with 
significant probability for all n sufficiently large, we get that 
dinil holds with 9 as in for < (5 < 1/2. □ 
As an application of the Corollary above, we estabhsh a 
new strong converse for SK capacity when the underlying rvs 

in) 

X)^ are i.i.d. Gaussian in n; for this model, the SK capacity 
was established in ll22l . The notions of e-achievable SK rate, 
e-SK capacity C(e) and SK capacity C are as in Definition |6] 
with condition ( ITTI i replaced by 

range(A') = {1, Lcxp(ni?)J}, (94) 

which rules out such rvs K as take infinitely many values. 



(") 



Proposition 15. When Xj^ 
AA(0,I]), 



is i.i.d. in n with X 



M 



C{e) 



2(k|-l) 



log 



nSiis. 



< e < 1. (95) 



Proof. That C(e) is no smaller than the right-side of 
follows from the achievability proof in 
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The proof of the reverse inequahty is along the lines of the 
proof of Theorem [12] and is obtained upon replacing the upper 
bound (l83]l by 

^ log - + 2pi?, 
n p 

and noting that Lemma[3]can be extended straightforwardly to 
an arbitrary rv V (with the explicit summations in the proof 
of that Lemma written as expectations), provided that the rv 
U is finite-valued. □ 
Proof of Theorem\r3\ In the manner of the proof of Theorem 
I2I it suffices to identify measurable sets Iq and £(i) C C, 
i e 2o> such that (l32])-(l34b are satisfied. Below we generalize 
appropriately the steps 1-3 in the proof of Theorem |2] 
Step 1. The following claim is an extension of Lemma [8] 

Lemma 16. Given measurable sets Ai € (Ti, 1 < i < k, for 
P in 



Pn,...,Y,\F {Ai X ... X A, I i) = n Py^lF {A, I i) , 

J = l 

Pp a.s. in i, (96) 

where PYi,...,Yk\F regular conditional probability on 

^yk^^k-^ conditioned on a{F). 

The proof uses the interactive property of the communication 
and is relegated to the Appendix. 

Step 2. Next, we identify the set Iq. The following technical 
observation will be used. 

Lemma 17. For every Aq e a'^ such that 
dP 



^(/) >o, 



it holds that 



yi,...,Yfc|F 



(97) 



(Ao I i) - / ^dP, PFa.s.ini 
dPF Jao dP 

(98) 

The proof is given in the Appendix. Denoting by To the set 

{y^- e 3^^' : < < 0}, let 

Ai^Ton {/ : Lj (y„F(/)) = L(y^) = M < j < fc} , 

Then, for Ai^ = AinF-^{i), (HHl and Lemma [17] imply 

5 f A \ ■\ dPp f dQi ~ 

aPp 



and by a reverse Markov inequality, that 



Pf (ii) > 1 - \/7T^. (101) 

Furthermore, for the set I2 of values i of F satisfying 

(102) 



^^(1) > S, 
dPp 



it holds that 



since 



Pf (I2) > 1 - <5, 



(103) 



dPp 



dPp ~ 
-^dPp 
15 dPp 



< S. 



Define In^IiH I2; follows from ( fToTT i and ( fT03] l. 
Step 3. Since Lemma |4] (i) applies to a countable set U = C, 
defining the nonnegative measure /i on £ as in ( |44] | for each 
i G Iq and using ( 1 100b . the sets £(i) obtained in (l45])-(l47b 
satisfy ( |32] ). Also, condition ( |33] | will follow from ( |46] | upon 
showing that 



(104) 



To do so, denote by Aj j the projection of Ai^i along the jth 
coordinate, 1 < J < fc- As before, the sets Aj ; are disjoint 
across I £ C. Then, Holder's inequality lfT6l impUes that 



> 



E^^-i y,|p(4iX...x4, |i)^ 



(105) 



dP 



rfP, Pp a.s. in i. 



where the previous step uses Lemma [16] The right-side of 
(1105b is bounded below by 

k 

EP^i.-.niF(Aili)^' ' 
since Ai^i Aj ^ x ... x ^f;, which by ( i99] l equals 



(99) 



E 

lec 



I "'^^ (i 




Upf 


i^,,, dP ; 



Below we restrict attention to the set of values of F for which 
( i99] l holds for every I £ C; this set has Pp measure 1 by ( i98] l 
since the set C is countable. Proceeding along the lines of the 
proof of Theorem [7] we define Ii as the set of those i for 
which 



From the definition of the set I2 in ( 1102b . the expression above 
exceeds 

k 

dQi 



PY^,...,Y,\F [Au I i) > 1 - VTT^. (100) 

Since L is an e-CR from F, it follows from ( i90l l. the fact that 
/:g(/)^0}).0, 



lec 

which is the same as 



dP 



-dP 



lec 



dQjdP/dP ^p' 
dP/dP , 



(106) 
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Since A; j C To, the sum in (IIO6I 1 is bounded below further lossless coding theorem for an i.i.d. source. In general, Propo- 



by 



E 

lec 

5 



E 

lec 



dP dP , 
cLqA 



= ^ EP>'i.-.v^|F(Ai|i)^ 
L/e£ 

Combining the observations above from ( |105t onward, we 
have 

fc 



> 



sition [18] implies a strong converse whenever the lower and 
upper bounds for R*{6) in (|107t coincide. This implication is a 
special case of a general source coding result in llT3l Theorem 
1.5.1], ITSll . where it was shown that a strong converse holds 
iff for rvs C/„ with pmfs /i„, the "lim-inf and "lim-sup" of 



Zn 



■ loe 



in /i„-probability coincide, i.e.. 



sup 

= inf{/3:liniM„(Z„>/3) = 0} 



(108) 



In fact, a straightforward calculation shows that the lower and 
upper bounds for R* (6) in (1107b are admissible choices of (3 
on the left- and right-sides of (1108b . respectively. 



which is the same as ( 1104b with a = 1/k 



□ 



B. General Models 



IX. Discussion 

A. General lossless source coding theorem 

Our Lemma |4] relating the cardinalities of large probability 
sets to Renyi entropy played a material role in the converse 
proofs. It is also of independent interest, and can be interpreted 
as a source coding result for a general source with finite 
alphabet U. Furthermore, it leads to the following asymptotic 
result. 

Consider a sequence of probability measures /i„ on finite 
sets Un, n > 1. For < 5 < 1, i? is a (5-achievable (block) 
source coding rate if there exists sets V„ C Un satisfying 

Ain(V„) > 1 - <5, 
for all n sufficiently large, and 

limsup — log \Vn\ < ^• 
n n 

The optimum source coding rate R* [5) is the infimum of all 
such ^-achievable rates. 

Proposition 18. For each < (5 < 1, 
lim limsup —i7(j(/x„) < R*{5) < lim \imsnp —Ha{iJ,„)- 

all n n Qfl „ n 

(107) 

Corollary. If /i„ is an i.i.d. probability measure on Un = 
U X ... X U, then 

R*{5) = H{^i^), 0<(5<1. 

Proof. The Proposition is a direct consequence of Lemma |4] 
upon taking appropriate limits in (l2ll and (l22T i with Un in the 
role of U. The Corollary follows since for i.i.d. yU„, 

Haifin) = nHaifJ-i) and lim Ha{ni) = 

a— >1 

□ 

Note that the Corollary above is proved without recourse 
to the AEP. Moreover, it contains a strong converse for the 



The description of the optimum query exponent in Defini- 
tion |4] can be refined to display an explicit dependence on e'. 
Let E*{e,e') denote the optimum query exponent for fixed 
< e, e' < 1. Our proofs establish E*{e,e') equals the right 
side of ^ for e' < (1 - y^)^ (see (ED)- For e' > 1 - e, 
as suggested by a reviewer, the following construction of L 
renders E*{e,e') unbounded: Choose L ~Q with probability 
(1 — e) and uniformly distributed on a sufficiently large set 
with probability e. For the remaining values of e, e', E*{e, e') 
is not known. 

A less restrictive model for querying than that in Section 
mi can be considered, allowing general queries with binary 
answers. Such a query strategy can be represented as a search 
on a binary tree whose leaves correspond to the values of the 
CR L. The query strategies considered in this paper correspond 
to the case where the search tree is a path with leaves attached 
to each node. For a general tree model, our results can be 
adapted to show that the maximum number of queries that 
can be inflicted on a querier grows only linearly in 71 at a rate 
that is equal to the expression for E* in (|5]). 

We remark also that allowing randomness at the terminals 
in A4 for interactive communication and CR recovery, does 
not improve the optimum query exponent. Such randomization 
is described by mutually independent rvs Wi, Wm, where 
each Wi is distributed uniformly on the (finite) set {1, ...,Wi}, 
and the rvs Wi, ...,Wm are independent of X"^. The claim 
of the remark is seen from the converse result in Theorem 
Indeed, the assertion ( fSSl l of Theorem |9] remains unchanged 
upon replacing Y, by {Y,, Wi), i e M, 9o by 6*0 {UieM ^i)' 
and 9b<= by 63-= (nieB<:^i)' B £ B; and observing that 
in ( l54b . the Wi- terms cancel in the numerator and the 
denominator 

Finally, Lemma [3] which considered rvs U, V, can be used 
to characterize the optimum query exponent T* for a family 
of finite-valued rvs {Un,Vn}^^i with associated probability 
measures {Pn}^i (which are not necessarily consistent). 
Here, P* is described analogously as E* in Definition |4] An 
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application of Lemma [3] yields that 



r* > P„-liminf ■ 



r* < P„- limsup ■ 



-logPc/„|y„ {Un I Vn) 



where the first and second limits above equal, respectively, the 
left- and right-sides of (|lQ8t with = P„ and 

_ -l0gP^„|y„(£/„ I Vn) 
— ■ 

n 

Appendix 
Proof of Lemma [76] 

For \ <l<r,\<i<k, denote by the interactive 
communication preceding Fjj, by Fjj the rv (Fij, 'i'(j), and by 
i; j a realization of F; j . Without loss of generality, we choose 
a version of Pyi ip that satisfies 



Py.|F,, (F-/(iy)^ I j = 0, Pf,, a.s., (Al) 

for all 1 < / < 7, 1 < j < fc. The following property of 
interactive communication is pivotal to our proof: For each 
^'j' '^^i'^^'ij^ ^ product set, i.e.. 

We prove the claim by induction upon observing that Py^ip^^ 
can be obtained by conditioning Vyk^i^^ . on the rv Fij. 
Formally, denote by a^(i^^ = cri(j;~) x ... x <Tk{i^j) the 

cr-field induced by a'' on A[ x ... x AJ,, and by a (^Fij{-, i;")^ 
the smallest sub-cr-field of cr^i^j) with respect to which Fij 
is measurable (for i^j fixed). Using (lAll i. we choose a version 
of Py*;|f such that for each \ < I < r and 1 < j < fc, 
Py"|F,j (■ I Uj) is the regular conditional probability on the 
probability space 

[a[ X ... X A';,, a^(i-), Py.|$,^ (• I 
conditioned on a (^Fij{-,i^j)j. Specifically, 



where the underlying cr-field for the conditional expectation 
is a^{iY-). For this version of Pytip, we show below that if 
(|96] l holds with in the role of F, then it holds with F(_, 
in the role of F. Lemma [16] then follows by induction since 
^ holds with F 0. 

It remains to prove the assertion above. To that end, for 
B e ^ij, denote by F^J^ {^B, i^^ the set 



With an abuse of notation, we do not distinguish between the 



sets Fi- yB, iijj and its cylindrical extension 
yi X ... X X ... X yk. 



Then, using the notation Q - and Q*- , 1 < t < k, for the 



probability measures Py'^l^ij I hjj and Pyt|$,^ 
1 < t < k, respectively, our induction hypothesis states 



QiijiAi X .... X Ak) = Yl Qhji^t), At£at, l<t<k. 

t=i 

(A3) 



It follows that 



LAiX...xAi 



LAinA;x...xAfcnA'j^ dQi-- 



where the first equality uses (lAlb and the second uses (IA3b . 
Defining 

Phi^) =H.;;[^A\a(Fl,{■^^i^))_ , 

Aeatiiij), l<t<k, 

we have from ( IA4l i that 

If,-U 



\{Pl{A^f^A',)dQ 
where the second equality uses (IA3b . Thus, by (IA2b . 

Pyfc|F,^ (Al X ... X Afc I iij) 

fc 

= n 4 (At n 4), Pf,, a.s. in Uj. (A5) 



t=i 



Since by dSB Pi*, (A^) = 1, 1 < t < fc, it follows from ([A5]l 
that 



= p/,(A,n4) 

= Py-IF,, (A'l X ... X X At X A;+i X ... X A'fc | i,,) 
= Py.|F,, {At I i;,). 



The previous observation, along with (IA5b . implies that 
holds with Fij in the role F. □ 

Proof of Lemma \T7\ 

It suffices to show that the right-side of ( i98] l constitutes a 
version of Ep [1^^ | cr(F)], i.e., 

F-i(-B) "'B VJAo C^Pf "P / 

(A6) 
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for every set B in the range cr-field of F. To show that, we 
note for every A E a'^ that 



/ lAdP= f Py\f{A I z)PF{dz) 
Jf-i(b) Jb 



where the previous step uses the assumption (|86> . Using 
Fubini's and Tonelli's theorems to interchange the order of 
integrals in iA7\ . we get 



F"i(B) 



which further implies 



/ ^PF(dz), Pa.s., (A8) 
Jb dP 

since the set A € was arbitrary. Next, for every B in the 
range cr-field of F, it follows from (IA8I 1 and WT\ that 



lF-i(i 



lA„dP= / lF-i(B)rfP 
F-i(B) JAa 

1 



-l^-HB)dP 



Ao dP/dP 

Jao dP/dPjB dP ^ ^ 

dQ, 



A„ JB 



dP 



■PF{dz)dP. (A9) [23] 



The claim iA6i follows upon interchanging the order of 
integrals in ( |A9] l. □ 
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